feat(search-pipeline): extract root types from the dataset registry - #698

Extract configured root types from the dataset registry instead of from each dataset’s own distribution.

A dataset’s description is governed by a different application profile from the objects it contains, and it lives in the register – nothing obliges a publisher to describe its own dataset inside its dump. So a Dataset root type extracted from the distribution yields documents for the few publishers that self-describe and nothing for the rest, leaving every reference resolved through it unlabelled.

Same CONSTRUCT generator, same framing, same projection, same writers – only the source differs. The extraction generator, the projection, the writers and SearchType are untouched.

Changes

@lde/pipeline

• StageOptions.sourceFor – a per-stage source override, (dataset, distribution) => Distribution. One hook, applied once at the top of Stage.run, so the item selector and every reader are handed the same substitute: a stage that reads elsewhere must select there too, or its roots and its quads come from different sources.
• SparqlItemSelector now honours Distribution.namedGraph, which only the reader applied. Without it a registry-sourced stage selects the whole catalogue instead of the dataset in hand. The graph scope and the page window are now derived per select() call rather than mutated on shared state, so concurrent selections over different datasets cannot see each other’s.

@lde/search-pipeline

• registrySource(endpoint) – a sourceFor that reads a fixed registry endpoint, scoped to the graph the dataset in hand names.
• SearchStageType.sourceFor passthrough, and searchIndexerPipeline’s registryTypes: { endpoint, names } for the convenience path. A name the schema does not declare throws at wiring time.

@lde/search-indexer

• REGISTRY_ROOT_TYPES (whitespace- or comma-separated type names) alongside REGISTRY_ENDPOINT. Unset means every type reads the distribution, exactly as before.

Why graph scoping carries the design

A register holds every registration, so an unscoped registry stage would re-index the whole catalogue once per dataset processed. Scoped to the dataset’s graph, one pass sees exactly one registration.

Verified against the live NDE Dataset Register (2731 registrations):

• every registration lives in a graph named exactly after its dataset IRI, so the dataset in hand names its own graph – no lookup, and FROM <dataset-iri> is the existing Distribution.namedGraph mechanism;
• a registration is self-contained in that graph: 2731/2731 dataset→publisher pairs carry the publisher’s foaf:name inside it, and 2730/2731 type the publisher foaf:Organization there too – so selectByClass works unchanged for a Publisher root type, and the publisher hop resolves from the declared paths with no CBD rule or predicate list;
• the register asserts dcat:Dataset in the dataset’s graph and schema:Dataset in the registrations graph, so a graph-scoped dcat:Dataset selector cannot reach the registrations graph at all – and excludes the 775 subjects that are schema:Dataset only (registrations never crawled), which a schema.org-rooted selector would ship as content-free documents.

Routing stays in configuration rather than in the schema: a SearchType is defined by its class, not by where its triples come from, so the same declaration serves a deployment that sources it differently.

Fix #695